Split pragmatics into presuppositions and scalar implicatures #2938
Conversation
|
Hi @weiqipedia, for your info. |
| ) | ||
| # Split "True or False" into ["True", "or", "False"] | ||
| choices = row["choices"].split() | ||
| choices_translated = row["choices_translated"].split() |
There was a problem hiding this comment.
Does this work consistently across every (supported) language?
There was a problem hiding this comment.
That's a good question! For now we only have Indonesian (and Tamil), and this splitting and taking the first and third index of the list does work for both languages. But just FYI, this will not work for Thai because of the lack of spaces, and we'll have to use something more similar to your suggestion of " or " (but we will not be having Thai any time soon)
| if self.language not in self.prompts.keys(): | ||
| raise (Exception(f"Unsupported language {self.language} - supported languages are {self.prompts.keys()}")) | ||
| else: | ||
| self.prompt_components = self.prompts[self.language] | ||
|
|
||
| def download_dataset(self, output_path: str): | ||
| BASE_URL = "https://raw.githubusercontent.com/aisingapore/BHASA/main/lindsea/" |
There was a problem hiding this comment.
Optional: You can pin this to a specific commit githash so that future changes to the git won't cause this scenario to change. e.g.
BASE_URL = "https://raw.githubusercontent.com/aisingapore/BHASA/10e34008e8142bef400cf8ffab15b2b6aaf3aa7f/lindsea/"
raileymontalan
force-pushed
the
lindsea_pragmatics_scenario_split
branch
from
0ab8fc3
to
3d88380
Compare
| dataset = pd.read_json(target_path_file, lines=True) | ||
| datasets = [] | ||
| for subset in self.subsets: | ||
| URL = f"{BASE_URL}{self.language}/pragmatics/pragmatic_reasoning_{subset}.jsonl" |
There was a problem hiding this comment.
nit: URL should be lowercase (it is not a constant)
No description provided.